Papers with contrastive learning methods
SkipCLM: Enhancing Crosslingual Alignment of Decoder Transformer Models via Contrastive Learning and Skip Connection (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing contrastive learning methods for cross-lingual alignment are not effective for multilingual machine translation tasks. |
| Approach: | They propose a method that augments contrastive learning for cross-lingual alignment with a trainable skip connection to preserve information crucial for accurate target language generation. |
| Outcome: | Experiments with XGLM-564M on the Flores-101 benchmark show that the proposed method preserves crucial information crucial for accurate target language generation. |
LAGCL4Rec: When LLMs Activate Interactions Potential in Graph Contrastive Learning for Recommendation (2025.findings-emnlp)
Copied to clipboard
Leqi Zheng, Chaokun Wang, Canzhi Chen, Jiajun Zhang, Cheng Wu, Zixin Song, Shannan Yan, Ziyang Liu, Hongwei Li
| Challenge: | Traditional contrastive learning methods treat negative feedback as equally hard or easy, ignoring informative semantic difficulty during training. |
| Approach: | They propose a framework leveraging Large Language Models to Activate interactions in Graph Contrastive Learning for Recommendation. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on multiple benchmarks. |
An End-to-End Contrastive Self-Supervised Learning Framework for Language Understanding (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to learning data representations using contrastive learning perform data augmentation and contrastive training separately. |
| Approach: | They propose a framework that performs data augmentation and contrastive learning end-to-end . they propose to combine data augmented with text encoders to optimize for contrastive training . |
| Outcome: | Experiments on GLUE and Gururangan datasets show the proposed framework is effective in NLP. |
FNSCC: Fuzzy Neighborhood-Aware Self-Supervised Contrastive Clustering for Short Text (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Short texts pose significant challenges for clustering due to semantic sparsity, limited context and fuzzy category boundaries. |
| Approach: | proposed framework incorporates neighborhood information at instance and cluster levels . a cluster-level framework introduces fuzzy neighborhood-aware weighting . |
| Outcome: | The proposed framework outperforms state-of-the-art models on short texts . it excludes neighbors from negative sample set to enhance inter-cluster separability . |
Scaling Sentence Embeddings with Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods based on contrastive learning have generated high-quality sentence embeddings. |
| Approach: | They propose a method to enhance LLM performance on sentence embeddings with a one-word limitation. |
| Outcome: | The proposed method outperforms contrastive learning methods on sentence embeddings without fine-tuning and with fine-untun. |
BERT-BC: A Unified Alignment and Interaction Model over Hierarchical BERT for Response Selection (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent performance boosting for dialogue response selection task achieved by Cross-Encoder based models is limited and the learned models have poor generalization capability in realistic scenarios. |
| Approach: | They propose a model that combines the representation-based Bi-Encoder and interaction-based Cross-Encoding to achieve better semantic representation. |
| Outcome: | The proposed model can achieve state-of-the-art performance on three benchmark datasets for multi-turn response selection. |
C3LPGCN:Integrating Contrastive Learning and Cooperative Learning with Prompt into Graph Convolutional Network for Aspect-based Sentiment Analysis (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that graph convolutional networks (GCNs) can model syntactic information but incorrect syntaktic structure may introduce additional noise. |
| Approach: | They propose a graph convolutional network which integrates Contrastive Learning and Cooperative Learning with Prompt into GCN to alleviate the noise when modeling syntactic information. |
| Outcome: | The proposed model outperforms state-of-the-art models on three datasets and significantly outperformed existing models. |
Exploring Representation-level Augmentation for Code Search (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent data augmentations for code search are at the raw-data level, which requires additional code analysis and training cost. |
| Approach: | They propose a general format of representation-level augmentation that unifies existing methods. |
| Outcome: | The proposed methods can boost the performance of code search models on a large-scale dataset. |
A Contrastive Framework for Learning Sentence Representations from Pairwise and Triple-wise Perspective in Angular Space (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning sentence representations focus on constitution of positive and negative representation pairs and do not focus on training objective. |
| Approach: | They propose a new method to learn sentence representations using BERT-like pre-trained models . they use a pairwise discriminating power and a model to model the entailment relation of triplet sentences . |
| Outcome: | The proposed method outperforms the previous state-of-the-art on diverse sentence related tasks. |
Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have become integral components in various autonomous agent systems. |
| Approach: | They propose an exploration-based trajectory optimization approach that allows agents to learn from their exploration failures. |
| Outcome: | The proposed method outperforms baseline methods on three complex tasks by a large margin. |
An Information Minimization Based Contrastive Learning Model for Unsupervised Sentence Embeddings Learning (2022.coling-1)
Copied to clipboard
| Challenge: | Recent contrastive learning methods keep positive pairs similar and push negative pairs apart, which leads to redundant information in sentence embeddings. |
| Approach: | They propose a contrastive learning approach which maximizes mutual information and minimizes the information entropy between positive and negative instances. |
| Outcome: | The proposed model outperforms all previous competitors on supervised and unsupervised tasks. |
Differentiable Data Augmentation for Contrastive Sentence Representation Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | a contrastive learning framework is used to fine-tune pre-trained language models with unlabeled sentences or labeled sentences. |
| Approach: | They propose a method that makes hard positives from unlabeled sentences . they use a prefix attached to a model to allow for differentiable data augmentation . |
| Outcome: | The proposed method yields significant improvements over existing methods under semi-supervised and supervised settings. |
Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Sentence embedding is essential for many NLP tasks, but reliance on manual labels limits scalability. |
| Approach: | They propose a method for controlling the generation direction of large language models in the latent space by integrating ranking information and semantic information. |
| Outcome: | The proposed method achieves new SOTA performance with a modest cost in ranking sentence synthesis. |